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PAT99310 PCT 

Comrnunication terminal handling messages including graphics. 

BACKGROUND OF THE INVENTION 
Field of the Invention 

5 The invention relates to the transmission of graphics as a part of the 
messaging between communication terminals. 

Description of the Prior Art 
On the Internet address http://www.kessler-design.com there is provided a 

10 graphic editor for generating a graphic picture on a Personal Computer in a 
predetermined formatted file, and to route this file via the Internet to a server 
where the message, based on the "Smart Messaging Concept" developed by 
the applicant is generated based on the received file. From here the message 
is routed to the cellular phone via an SMS gateway & SMS interworking 

15 Mobile Switching Center (MSC). 

Philips Consumer Communications has exposed a new phone SAWY™ 
GSM phone in Cannes 23. February 1999. This phone will be equipped with 
a message application using graphics. It is passible to include "clip art 11 like 
20 graphics in messages exchanged between two SAWY™ phones. Basically 
this means that the "clip art" like graphics are handled as predefined 
characters and the character is transformed to a hexadecimal code in one 
phone, transferred back to the same predefined character in the receiving 
phone. 

25 

SUMMARY OF THE INVENTION 
An obj e ct of the T hejnvention r e to provid e provides a method for handling 
transmission of messages including graphics between communication 
terminals. 

30 

This purpose is obtained by a method for handling messages transmitted 
between communication terminals via a wireless network, and comprising 
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generation of a compound message including a text part and at least one 
graphical icon part and transmission of the message via the wireless network. 
The handling of the compound message generation comprises steps of 
reading of a user inputted text part and converting the inputted text into in a 
5 predefined message text format, adding a graphical part to the message, said 
graphical part i nclud es including a record for each of said at least one 
graphical icon part in a graphical format, and adding information in the 
message defining the position of said at least one graphical icon part in the 
text part. Hereby the phones will be able to include a real graphical file into 
10 the message and the receiving phone will be able to decode the graphical 
icon without knowing the pattern in advance. 

The invention dees-furthermore felate -relates t o a communication terminal for 
handling messages and having a controller, a transceiver for communicating 

15 with a wireless communication network, and a user interface through which 
the user operates the terminal, said -the user interface includ e s including a 
display. The communication terminal furthermore comprises a message 
editor application by means of which the user is allowed to generate a 
compound message including a text part and at least one graphical icon part; 

20 said controller generates a message for transmission via said transceiver. 
The message includes a text part in a predefined message text character 
format, a graphical part including a record for each of said at least one 
graphical icon part in a graphical format, and information in the message 
defining the position of said at least one graphical icon part in the text part. 

25 The graphics used in the message may be found in a clip art like gallery 
including a plurality of graphical icons, or manually entered by the user by 
means of a graphical editor. 

The invention does-furthermore rolate relates t o a message format including 
30 a text part and at least one graphical icon part. The message includes a text 
part in a predefined message text character format, a graphical part including 
a record for each of said at least one graphical icon part in a graphical format, 
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3 

and information in the message defining the position of said at least one 
graphical icon part in the text part. Hereby an existing text message carrying 
concept such as e ^f or example SMS messaging in the GSM system will-bo 
isable to transport graphics when the receiving and transmitting terminals 
5 have respective graphical editors/readers for coding and decoding the 
messages. 

According to the preferred embodiment this information is handled by the 
Smart Messaging Concept. This concept was disclosed by the applicant at 

io the Cebit fair in March 1997 (Press release 20. March 1997) and discussed 
in details the document: "Smart Messaging Specification", Revision 1.0.0; 
September being available on the Internet address URL 
www.forum.nokia.com/nf/concepts/smart/main.html. From this Internet 
address another document "Narrow-Band Sockets Specification", Revision 

is 1 .0, March 7, 1997 may be downloaded. This document describes the 

Narrow-Band Socket (developed by a joint effort by of Intel and Nokia) that 
enables applications to access various network data bearer services using a 
standard socket interface. 

20 According to the invention it is very important to improve the character of 
sending messages in a cellular phone network from being a pure text 
messaging system to become a more sophisticated concept including 
graphics into messages and thereby attracting younger people who want to 
generate tags or symbols identifying themselves when sending messages. 

25 

In addition to offering a huge amount of templates or predefined graphics the 
inv e ntion s w ill -invention a ccording to the preferred embodiment offer offers 
the user a built-in graphics composer application by means of which the user 
is allowed to create graphics himeelf-directly on the phone. The user will also 
30 not need to buy a PC interface cable, to be able to create graphics on a PC. 



PAGE 30/39 1 RCVD AT 1 1/2412004 2:03:57 PM [Eastern Standard Time] * SVR:USPT0-EFXRF-1/1 * DNIS:8729309 * CSID:7033126666 * DURATION (mm-ss):1D»26 



11/24/2004 14:05 7033126686 BR I NT PAGE 31/39 



4 

According to the invention it is essential to offer sending and receiving 
graphics directly from one phone to another. Furthermore the graphics are 
actually included into the message text. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

For a better understanding of the present invention and to understand how 
the same may be brought into effect reference will now be made, by way of 
example only, to accompanying drawings, in which:- 

10 Fig. 1 schematically illustrates a preferred embodiment of a hand portable 
phone according to the invention. 

Fig. 2 schematically shows the essential parts of a telephone for 
communication with a cellular or cordless network. 

15 

Fig. 3 illustrates the message reception scenario according to the invention. 

Fig. 4 illustrates the message transmission scenario according to the 
invention. 

20 

Fig. 5 shows a display sequence of a phone receiving a message including 
graphics according to the invention. 

Fig. 6 shows a display sequence of a phone inserting graphics into a 
25 message for transmission according to the invention. 

Fig. 7 shows a display of a graphical editor of a phone for manually entering a 
new graphical icon according to the invention. 

30 Fig. 8 schematically shows the essential parts of a GMS editor/reader 
according to the invention. 

i 
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5 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Fig. 1 shows a preferred embodiment of a phone according to the invention, 
and it will be seen that the phone, which is generally designated by 1, 
comprises a user interface having a keypad 2 f a display 3, an on/off button 4, 
5 a speaker 5, and a microphone 6 (only openings are shown). The phone 1 
according to the preferred embodiment fs adapted for communication via a 
cellular network, but coufd have been designed for a cordless network as 
well. 

10 According to the preferred embodiment the keypad 2 has a first group 7 of 
keys as alphanumeric keys, a soft key 8, a clear key 9, and a navigation key 
or a scroll key 1 0. The present functionality of the soft key 8 is shown in a 
separate field in the display 3 just above the key 8. 

15 Fig. 2 schematically shows the most important parts of a preferred 
embodiment of the phone, eaid -these p arts being essential to the 
understanding of the invention. The preferred embodiment of the phone of 
the invention is adapted for use in connection with the GSM 900MHz and 
GSM 1800 MHz network, but, of course, the invention may also be applied in 

20 connection with other phone networks. The processor 18 controls the 

communication with the network via the transmitter/receiver circuit 19 and an 
antenna 20 that will be discussed in details below. 

The microphone 6 transforms the user's speech into the analog signals, the 
25 signals formed thereby are A/D oonverted in an A/D converter (not shown) 
before the speech is encoded in an audio part 14. The encoded speech 
signal is transferred to the processor 18, which i^is supports the GSM 
terminal software. The processor 18 also forms the interface to the peripheral 
units of the apparatus, including a RAM memory 17a and a Flash ROM 
30 memory 1 7b f a SIM card 1 6, the display 3 and the keypad 2 (as well as data, 
power supply, etc.). The audio part 14 speech-decodes the signal, which is 
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transferred from the processor 18 to the earpiece 5 via a D/A converter (not 
shown). 

Receiving messages Including graphics. 

5 The ability to receive and send graphics via messages will require the 

features of storing a large amount of pre-specified graphics inside the phone 
(Clip-art / graphics templates), the ability to insert graphics into a message 
being composed, a method to browse the huge amount of graphics and a 
possibility to save received graphics, and to afterwards use them in message 

10 composing. According to the preferred embodiment of the invention it is 
furthermore possible to generate new graphics on the wireless 
communication terminal or the phone 1. 

According to the preferred embodiment when the user receives a message, 
15 the reception will be indicated in Idle mode of the phone 1 , like the first image 
of figrFig. 5. A header 50 Includes some icons informing the user about 
erg rfor example the status of phone settings, time and un-read messages 
present in the phonebook. Furthermore it is seen from the first line that a new 
message is received, and the second line indicates the phone number or the 
20 name of the sender if this can be found in the phonebook based on the 
phone number of the sender. The next two lines indicate the start of the 
message, if several messages are received, the initial text from the newest 
message will be displayed. The functionality of the soft key 8 will 
advantageously be "read" as seen from the first display. Pressing this softkey 
25 8 allows the user to read and edit the entire message as shown in the second 
display of-fei rFiq. 5. 

In this case, the user has chosen to read the message, simply by pressing 
the soft-key 8 having the (t Read" functionality. This will directly enter the 
30 message viewing as seen in the second display of feh Fig. 5. Now the 

functionality of the soft key 8 changes to "option", and by pressing this key 8 
the user gets direct access to e.g. delete, edit or forward the message. By 
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using the scroll key 10 the user may scroll through the message as seen In 
the third display from the second and third display it is seen that the header 
50 now identifies the sender of the message. Furthermore it is seen that the 
message includes graphics 51 formed as two Dolphins (the trademark of the 
5 Danish operator Sonofon). 

Graphics can basically be of two types: "Large graphics" - as the Dolphins 
shown in the third display; and "In-line graphics" that means graphics with 
maximum height of that corresponding to the height of the text, e.g. 8 pixels. 
10 "In-line graphics" are placed in a text line and displayed as a text character. 
However the handling of the In-line graphics in the message will be similar to 
the handling of Large graphics. 

The fourth display shows a message including "In-line graphics" 52. The 
15 width of the graphics can basically be up to the width of a text line. However a 
maximum width of two-three ohanaot e r characters makes it easier to handle 
the message display. If the user (in the second, third or fourth display) now 
wants to save the graphics h e can pr ess the soft key 8 is pressed having the | 
"Options" functionality and select "user graphics" in the option list displayed in 
20 the fifth display. 

When the "User Graphics" option is selected a list (page wise) of all the 
graphics stored in tine message will be given. In the message shown in 
fiO rRg. 5 second and third display only a single graphic 51 is included- The 
25 default options on the soft-key 8 will now allow the user to store the graphics 
directly to the "user graphics" gallery of the phone 1 for later use of the 
graphics. 

Fig. 3 illustrates the message received scenario. In step 100 a message is 
30 received and in step 101 the GMS -GRAPHICAL MESSAGING SYSTEM 

(GSM) reader 30 checks whether the received message is in the GMS format 
or not. If not the GMS reader 30 just starts to wait for the next message. 
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8 

Otherwise the GMS reader 30 reads the message and starts to identify the 
text part and possible graphics in step 102. When this is done the GMS 
message is displayed for the user in step 103. In step 104 the user is allowed 
to store the received graphics and this is done in step 105 upon request from 
5 the user. In step 1 06 the display of the messages remains until the user 
wants to dose the message. 

Irvfig rFlg. 4 it is shown that the GMS editor starts in step 110 when the user 
wants to enter a GMS message. In step 1 1 1 the entered text is read and 

10 when the user wants to enter graphics he -the user c an indicate this in step 
1 12. Then te sthe user w ill have the choice (in step 113) of entering a new 
graphic by means of using the graphical editor 35 (step 1 14) of by recalling a 
graphical icon stored in one of the memories 32 or 33 (fig. 8) in step 115. 
When the user does not want to enter further text or graphics fte -the user can 

15 request transmission in step 116 whereby the GMS message is transmitted in 
step 1 17 and the application is closed. In stepl 16 the user will be allowed to 
close the application without transmitting the message. 

Sending messages including graphics, 

20 The user has the ability to insert graphics into the message. The user can 
insert graphics from both a pre-defined Clip-art directory, and from the "user 
graphics* 4 gallery. In the first display of fi ^Fip. 6 the user has started to 
compose a message. The soft-key 8 now allows the user directly to send the 
message, but by using the scroll key 10 alternative functionalities, including 

25 insert of graphics in the option called "Insert Graphic" as seen in the second 
picture of fig rfiq. 6. By selecting the "Insert Graphics 1 * option the user will be 
given a list of graphics groups, like: User Graphics, Animals, Business, 
Cartoons etc. as shown in the third display of figrHg. 6. 

30 This list will contain logical groups of graphics from the clip-art database, and 
- as the first item - the complete list of the graphics stored by the user. The 
list will also contain an item called "In-line icons", which will contain icons that 



PAGE 35/39 1 RCVD AT 11/24/2004 2:03:57 PM [Eastern Standard Tune] * SVR:USPT0-EFXRF-1/1 * DNIS:8729309* CSID:7033126666 1 DURATION (mm-ss):1D»26 



11/24/2004 14:05 7033126S66 BR I NT PAGE 36/39 
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can be included in the text lines, preferably having a maximum height 
corresponding to 8 pixels. Preferably, the list will also contain CLI icons, start- 
up graphics etc. Selecting any of the items in this list, will give the user a list 
of ail the graphics in the group. Selecting e ^for example the item called 

5 "Animals" could give one of the graphics shown in the fourth or fifth display of 
figrFig. 6, Scrolling down with the scroll key 10 will show the next graphic in 
the "Animals" group. Pressing the soft-key 8 having the functionality "Insert", 
will insert the graphic into the message text at the position of the cursor 53 
(first display of foh F\q± 6), The graphic is now inserted into the text, and the 

io editor is ready for insertion of a new word to be added after the graphic as 
shown in the sixth display of fig rFifl. 6. 

According to the invention it is very important to improve the character of 
sending messages in a cellular phone network from being a pure text 
15 messaging system to become a-more sophisticated conc e pt to permit 

including graphics into messages and thereby attracting younger people who 
want to generate tags or symbols identifying themselves when sending 
messages, 

20 The graphics editor basically allows the user to edit and define any pixels in 
the display. The pixel editor display is displayed in tig rFig. 7 and each pixel in 
the final drawing consists of 3 x 3 pixels in the editor. The current position of 
the cursor is indicated with a hollow pixel 70. A small preview 71 of the final 
drawing can be shown in the lower right comer of the display. By using the 

25 soft key functionality "Options", the user can e ^or example invert a specific 
pixel. The basic soft key functionalities under "Option" could allow the user to 
simply invert pixels individually, but functions like "Circle", "Insert Text", 
"Reverse all" will also be included. The cursor or hollow pixel 70 is moved In 
the display by means of navigation means. The navigation key 10 only allows 

30 the user to navigate in two directions, and therefore it is preferred to dedicate 
the "2\ "4", "6" and 4, 8 B keys for moving the cursor in the four directions of the 
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10 

display. However an improved embodiment could include a four directional 
scroll key. 

Animations. 

5 Like with messages and graphics, the graphical reader could offer the user a 
variety of pre-defined animations, which the user can attach to different 
"events" in the user interface. These events oould e^or example be when | 
entering a menu item, start-up animations, when an incoming call appears 
(for example CLI Group dependent), when the keyguard is activated or de- 

io activated, etc. 

It is evident that the invention supports a scenario in which the user creates 
hfe -the user's own animations, e ^for example on a Personal Computer and 
then downloads the animation to the phone T a^whi^lisused just like the 
is pre-defined animations. The user can just send the animation to another 
user. 

Basically these animations are a sequence of pre-stored graphics that are 
displayed as a sequence. An animation generator has to be able to identify 

20 and access the individual graphics in a memory. Furthermore a set of 

parameters has to be present for the individual animation and the parameters 
must contain an identification of a start event, such as start up of the phone, 
detection of an incoming call etc. Furthermore the refreshing time or period 
has to be Identified le- that is the time one graphic is displayed before it is 

25 replaced by the next graphic. Finally the duration of the animation has to be 
given and this will typically be when the full animation has been played once, 
or when the graphics in the full animation have been played as an endless 
sequence for erfrr example two or three times. 

30 Graphical Messaging System. 

The Graphical Me s ooging Syotom ( GMS) is able to transport text, pictures 
and animations as one entity in messages handled by e-^rfor example the 
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tl 

Nokia Smart Messaging format. The Nokia Smart Messaging format is 
presently used in connection with the QSM messaging system and therefore 
the overall concept may be regarded as well known feF-toa mafvperson | 
skilled in the art. The format of the messages including graphics transmitted 
from one phone to another according to the preferred embodiment of the 
invention will be described below with reference to figrFig, 4. The message | 
format is based on the Nokia/Intel Narrow Band Socket (IMBS) specification. 
This format is already well known and used for a number of services already 
implemented in a wide range of Nokia products, e^-for example. Over The | 
Air (OTA) business cards, OTA ringing tone download etc. 

A GMS reader 30 is listening to a Narrow BAND Socket (NBS) port 5514 
decimal [158A hexadecimal] via the processor 18. The received message is 
displayed in the display 3 and temporarily stored in a memory 34. The reader 
15 30 identifies the text part and the graphics part of the message and the user 
Is as mentioned above, able to store the received graphics in a "User 
Graphics" memory 32 for later user. 

It is well known to use text editors in e^ rfor example GSM phones for 
20 generating a text for a message to be transmitted. A GMS editor 31 replaces 
this text editor and act -acts f ully as such one when handling text alone. The 
user is able to enter a text part by using the text entry means - preferably the 
alphanumeric keys, and when the user via the "Option" functionality in the 
soft key 8 starts to enter graphics he -the user c an get one precoded graphical 
25 icon from a memory 33 including a plurality of clip art like graphics - 

preferably included in a plurality of groups (animals, business, cartoons etc.) 
in order to ease the search for a specific Icon. Alternatively he-the user_can 
enter the desired pattern by means of a graphical editor 35 (display is shown 
in foh-Fig. 7) and store the manually entered graphic in a "User graphic" 
30 memory 32 and use the graphic in the GMS editor. The "User graphic- 
memory 32 includes manually entered graphics, graphics received In a 
message, or clip art like graphics amended in the graphical editor 35. 
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GMS format 

The actual GMS format will be based on the Nokia Smart Messaging format, 
but the actual format will be as follows: 

5 

< GMS-message> ::= <GMS-header>< GMS-item> + 

< GMS-header> ::= < GMS-Version> 

< GMS-Vereion> "= '0* ;'tdentifler for GMS version, current version is zero (0). 

<GMS-itam> ::=*00" <GMS-item-length> <IS08859-1-char>* 
10 | H 01 w <GMS^tem-l©ngth> <UNtCODE-char>* 

| "02" <GMS-ltem-length> <OTA-bitmap> 

| "03" <GMS4tem-length> <Ringing-tone-programming-!anguage> 
j <common«hex-digit> <common-hex-dlglt> <GSM-item-length> 
<unspecified-data> 

l5 ; Two hex characters who specify the GSM item type, if not 00.,.03. 

Tokens 04 ... FF are reserved. If one of these values Is present the 
application may either stop processing or skip that GMS-item. 1 
<unspecified-data> " a <default-char>* ; 'A stream of octets' 
<GMS-ltenrHength> :;- 
20 <common-hex-digit> <common-hex-digit> 

<common-hex-digit> <common-hex-dlgit> 
; 'Number of octets In the following field, in hexadecimal. 
Note that there are two octets per each Unicode character.' 
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